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ABSTRACT 


An assessed hotel service is necessary for tourists and everyone who is 
traveling, however currently it is still difficult to find recommended hotel 
information. The solution provided in this research is to propose a smart 
application that has been developed by implementing machine learning in it. 
The purpose is to build a sentiment review smart application by applying the 
sentiment analysis hybrid model of the best neural network (NN) algorithm 
model that has been optimized using genetic algorithms. To get the right 
model, the research method was carried out with experiments starting from 
the initial stages of conducting data preprocessing, tokenization, weighting, 
modeling experiments, and conducting the system evaluation stage to 
determine the success of the proposed model. The progress of the application 
development system is by using the prototyping model. SentiHotel is a 
sentiment application that was successfully built to provide a solution for 
tourists in assessing a hotel service. The software validation test is carried out 


using the blackbox method and the results show that the SentHotel 
application is in accordance with the expected result, all system functions can 
run properly. 
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1. INTRODUCTION 

The service level guality of a company engaged in the service sector is part of the criteria for 
evaluating a service user. Services that are beyond expectation will provide an increased level of trust in 
these service providers and will be worth the price, the more satisfying the service, the higher the service fees 
that must be paid by users. The hotel and lodging service sector currently provides enormous opportunities 
for these service providers, especially in the tourism sector. Assessment of the level of a hotel service 
satisfaction can come from various sources, one of which is a review comments of tourists or travelers who 
have used the lodging services [1]. 

The development of artificial intelligence (AI), especially machine learning, makes everything 
easier to apply. In accordance with its advantages, one part of ML is a supervised learning algorithm, which 
can be used not only for control and monitoring systems [2], but also in data science where it can be 
implemented in a data classification that applies in various fields, such as health [3], livestock and agriculture 
[4], [5], economy and industry [6], transportation [7], education, health [8] and others [9]. 
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The rise of social media which currently attract more and more user, provides a huge potential, 
especially for many companies. They no longer need to expose their products in a paid service advertisement. 
This benefit of social media also provides an advantage for data seekers, especially data that comes from 
comments from social media users which are used as a decision support. Data derived from these comments 
is used as a dataset to be processed using text mining technology. This technology allow data in the form of 
written text in a review and comments to become data that can be classified or predicted [10]. The use of 
sentiment analysis that comes from social media has been used by several previous researchers [11], [12]. It 
was used for the sentiment assessment of the relocation of the Indonesia capital city [13], as well as the use 
of real time social media bigdata for sentiment [14]. The sentiment analysis is also used to assess film 
reviews [15]. Another study is the application of sentiment by optimizing the accuracy of the support vector 
machine model to overcome hyper-parameters [16]. 

Building a trust from the service sector business is currently a top priority for these company 
owners. The consumers trust that are assessed from a service has a very significant effect on business 
continuity in this service sector. The lodging service business of a hotel is currently growing rapidly and is 
unique because it is almost said that the level of service determines the success or failure of the hotel. The 
more people feel that they are getting services that are worth the money paid, the more likely that person will 
have an impact in the future, since it is almost certain that the experiences they have will be shared with those 
around them either directly or through social media. Marketing through a review comment written by hotel 
customers will indirectly affect the failure of the hotel accommodation service business especially negative 
review, because prospective customer will see the review results from comments of hotel users; if the results 
of the review are positive sentiments, then it will affect their decision to stay at the hotel, and vice versa. 

Research on hotel and restaurant sentiment has actually been carried out by several previous 
researchers using various models and algorithms [17]-[20], however, almost all of these studies use English 
as input data as well as the data sets. Previous research that has been carried out in the search for a hotel 
service sentiment review model using the decision tree model produces the best accuracy rate of 88.54% 
[21], but it is still not optimal. The next research was carried out by applying a neural network model that had 
optimized parameter values and resulted in a classification accuracy rate of 88.99% [22]. Based on a model 
with a known level of accuracy, this study develops and builds a sentiment analysis-based application to 
classify hotel service ratings based on customer reviews using Indonesian text, slightly different from [23]. 
The basic contribution of this research is that it offers an intelligent sentiment analysis application SentiHotel 
which is developed by integrating a neural network model that has been optimized for parameter values using 
a genetic algorithm. This application was developed to make it easier for tourists and everyone who wants to 
find the best hotel recommendations with the best satisfying service. The contribution of this research is to 
develop a new application based on machine learning that has a better level of accuracy and can provide 
better benefits since the application proposed and developed can be accessed anywhere by everyone. The 
constructed classification in the system proposed consists of two sentiments; positive and negative sentiment. 


2. RESEARCH METHOD 
2.1. Dataset 

To build a system and applications, sentiment data for hotel review assessments obtained from the 
site google maps [24] with the keyword hotels in the Central Java region is used. The data used is in the form 
of text data which is processed in the data preprocessing process and then processed into a model applied to 
the application. An overview of the dataset used in the search for the built-in application model is as shown 
in Figure 1. The data used is Indonesian text data taken in 2019 which consists of a collection of negative 
assessment comments with a 1-2 stars rating and positive reviews with a rating between 4-5 stars. Overall, 
the number of review comments for this hotel rating is 414 reviews [25]. 


9/3 2 tahun lalu di G Google 1/5 2 tahun lalu di G Google 
Hotel baru yang menarik di kota tegal. Mempunyai 


— mahal, karyawan tidak ramah, kamar sempit, makanan 
fasilitas yang lengkap. Menu breakfast lengkap dan 


_ Ne tidak istimewa, and not rocemmended 
ada menu jajanan gorengan khas tegal/slawi yaitu 


tahu aci. Selain menu standar yang biasa ada,seperti 
tradisional dan western 


(a) (b) 
Figure 1. Sample dataset for the proposed model, (a) Positive comment review, (b) Negatif comment review 
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2.2. Proposed model in system 

The development of a hotel service sentiment analysis application in this study applies a previously 
obtained model, namely a model based on the neural network algorithm with predetermined parameters. The 
depiction of the model of the neural network that is applied is as shown in Figure 2 where the expected data 
output has two neurons. This means that the expected result is a classification of two different labels, namely 
positive sentiment or negative sentiment. The process of tokenization, filtering text, and weighting of text 
data is carried out during data processing stage. The depiction of the applied model adopts the research that 
has been done before, which is shown in Figure 2. Two classifications of review sentiment have been 
assigned in the model obtained, namely positive sentiment and negative sentiment. To get the accuracy of the 
model, data validation is used using k-fold cross validation with predefined parameters and produces the 
expected level of accuracy. 
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Figure 2. Framework of SentiHotel model used [21] 


2.3. Stages of the system development 

To build a system in the intelligent hotel sentiment analysis application, each stage is made using 
the Prototyping system development model. The stages carried out in this model are the system requirements 
analysis stage, building prototyping, protoype evaluation, system coding, system testing, system evaluation 
and system implementation. Each stage that is carried out is adjusted to the circumstances of the situation and 
conditions in the field, however it still follow the concept and is built according to planning. The depiction of 
the activities carried out in the development of this system in outline stages is shown in Figure 3. 





Figure 3. Model prototyping system development method [26] 
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3. RESULTS AND DISCUSSION 
3.1. Proposed model 

In this article, the proposed model is a neural network-based model that has been optimized by 
applying the genetic algorithm. The specified parameters are the values obtained after parameter optimization 
by using a genetic algorithm and selecting the best weight using information gain (IG). The NN model 
obtained in Figure 4 is an NN model that uses the sigmoid activation function used (1). 


1 


fx = (1) 


1-eX 





The model obtained consists of 10 input neurons, 7 hidden neurons, and 2 output neurons. In this 
model, there is 1 bias neuron in the input layer and 1 bias neuron in the hidden layer. The proposed model 
was obtained using the Rapid Miner Studio software analysis, Windows 10 operating system, Core 17 
processor, and 8 GB RAM. 
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Figure 4. Neural network model applied to the application 


The best parameters that have been obtained to produce this model are using the tournament 
selection type, training cycles=94, learning rate=0.8589114570.858911457, and momentum=0.586464677. 
The expected level of accuracy is obtained by using (2), (3) and (4). 


True Positive (TP)+True Negative (TN) 


Accuracy = = ee (2) 
True Positive (TP)+False Negative (FN)+ True Negative (TN)+False Positive (FP) 
a True Positive (TP) 
precision = ——£ ————__—_——___ (3) 
True Positive (TP)+False Postive (FP) 
True Positive (TP) 
Recall = (4) 


True Positive (TP)+False Negative (FN) 


Based on the experimental results in this model, the best level of accuracy for the classification of 
hotel service ratings is 88.99%. This level of accuracy is obtained based on the predetermined parameter 
combination settings. It has produced various combinations of accuracy level values, and the experimental 
results which are shown in Table 1 and Figure 5. 
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Table 1. The evaluation of the 


Evaluation Model 
best model uses several 


in sampling methods 

89 Sampling Accuracy (%) 
88,8 stratified 88,26 

shuffle 88,26 

88,6 Roulette wheel 88,99 
88,4 
88,2 

88 
87,8 





Accuracy (54) 


E stratified E shuffle Roulette wheel 


Figure 5. Evaluation of the best NN model 


3.2. System development design 

In the development of smart applications that are made, the modeling system is shown in the use 
case figure. In the figure, it shows that the user consists of two actors, namely admin and end user. These 
actors have their respective functions. The admin is given access to analyze text data that will be input 
including carrying out the processes needed to produce sentiment classifications starting from data 
preprocessing to producing sentiment classifications which will be forwarded into the hotel recommendations 
system with the best service. The end-user are slightly different. They have limited access where they are 
only capable of seeing hotel recommendations and hotel details. The use case depiction of the designed 
system is as shown in Figure 6. 


include» 


- 


Praprocessing Text 





i 


Admin ' «extend» Classification Sentiment 
«include» 
Hotel Recomendation 


End User ae 
Detail Info Hotel 


Figure 6. Use case diagram of smart application of hotel service sentiment SentiHotel 


3.3. System implementation 

The login page is a feature used specifically for admins who will enter the application. This login 
system can be accessed only by registered admin users, therefore not all users can enter the system. On this 
interface, there are two data input, username and password, which will be used as a user validation system if 
you want to enter the system. The image shown in Figure 7 shows the main page of the admin user. There are 
several features that are displayed on this main page, including accapitulations regarding the number of 
hotels reviewed, the number of visitor reviews, and user data. The main page access of the visitor section is 
also displayed in this feature. 
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Show 1 v entries Search: 
No Nama Hotel Kelas Alamat Sentimen 

o 1 PRIME BIZ kam YY JL Mayjend Sutoyo No.25 Pekauman Kec. Tegal Bar. Kota Tegal Jawa 
Tengah 52113 

oO 2 MARLIN PEKALONGAN Re Jl. Raya Wiradesa No.25 Mayangan Kec. Wiradesa Pekalongan Jawa | 
Tengah 51152 

a 3 KARLITA HOTEL kaka ti Jl. Brigjen. Katamso No.31 Tegalsari Kec. Tegal Bar.Kota Tegal Jawa 
Tengah 52111 

a| 4 SENDANG SARI BATANG LA A Atti Jl. Jendral Sudirman No.29 Kasepuhan Kec. Batang Kabupaten 
Batang Jawa Tengah 51216 

Oo 5 THE WINNER PREMIER HOTEL korri Jl. Jend. A. Yani No.9 MulyoharjonKec. Pemalang Kabupaten | | 
Pemalang Jawa Tengah 52312 

oO 6 FAVE CILACAP kkk Jl. Budi Utomo No.38, Sidakaya Dua, Sidakaya, Kec. Cilacap Sel., 
Kabupaten Cilacap, Jawa Tengah 53212 

o 7 DAFAM HOTEL be Bt eter: Jl. Dr. Wahidin Sudiro Husodo No.5-15, Dafam Cilacap, Sidakaya, 
Kec. Cilacap Sel., Kabupaten Cilacap, Jawa Tengah 53211 

o 8 MUTIARA HOTEL koiri JL Gatot Subroto No.136, Wanasari, Gunungsimping, Kec. Cilacap 





Tengah, Kabupaten Cilacap, Jawa Tengah 53224 


Showing 1 to 8 of 8 entries Previous Next 


Figure 7. Main admin page interface 


On this admin page, a recapitulation and a list of the names of the hotels being reviewed are also 
displayed, along with a “class” rating which contains an asterisk rating totaling 5 (five) stars. In the data list, 
the hotel address and the results of sentiment analysis are also displayed as well as an edit feature for other 
analysis processes. There are facilities to change the hotel data in this feature if there are any changes. Data 
change will adjust to the new data input. To provide an information obtained by the end-user as the user of 
this sentiment analysis, the main display containing information related to the hotel review sentiment is 
created. This page contains data regarding this application information as well as some information that can 
be displayed about a hotel service. This information is in the form of hotel data that can be selected according 
to the details of the data that have been input. To see details, the user just has to select "more info". The 
interface features of the main end-user page are shown in Figure 8. 

On the feature page of Figure 8, list of hotel names is compiled based on incoming data and the 
results of the analysis that have been carried out on the previous admin page. End-users can only access it 
without logging in to the system in this feature. If you want to know detailed information about the sentiment 
results from the hotel, then in Figure 8 you can see a detailed feature menu display that provides it. Figure 9 
shows detailed information about the hotel based on the results of the sentiment analysis that has been carried 
out. In this display, the information obtained is detailed hotel information as well as the address equipped 
with a map of the selected hotel address. In this feature, we can see the review sentiment display of this hotel 
which can be displayed graphically and can be seen on a monthly basis, as well as the comments that come in 
for this hotel review. 


3.4. System evaluation 

System evaluation is carried out to provide an idea that the applications that have been made are 
aligned properly. This testing process using the Black Box method that focused on system functions that are 
in process and have been made. The test results are shown in Table 2, Table 3, Table 4, Table 5, and Table 6. 
Table 2 shows the system testing carried out on the funnctional system login done by the admin. The system 
is tested using two functions alternately in this test. From the results, it is found that the functional login 
system runs as it should. 
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Figure 8. The end-user main page interface 


SENTIMEN Beranda Tentang Kontak Informasi ~ Search hotel... Q 


Selamat Datang! Di Website Analisis sentimen hotel, berdasarkan ulasan pelanggan pada Google Maps Review. Find out More! 





Analysis Reporting @ Beranda Report 


~ Sentiment Hotel PRIME BIZ ~ 


Dibawah ini adalah laporan terhadap analis sentimen masyarakat terhadap hotel PRIME BIZ yang memiliki alamat di JL Mayjend Sutoyo No.25 Pekauman Kec. Tegal Bar. Kota Tegal 


Jawa Tengah 52113 diambil berdasarkan komentar di ulasan google maps, dimana data mentah masih dalam bentuk kalimat-kalimat yang sulit dipahami sistem namun kemudian di 


olah dengan menghilangkan kata yang tidak perlu dan memisahkan kata dasar dari imbuhanya. 
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Piechart Sentiments — $$ 
No Komentar 
1 checkin 2 hari hari pertama checkin kecewa tunggu kamar siap lama stengah jam padahal sdh booking apa 
booking masuk kamar pengap bgt kaya gudang hari telpon pihak hotel kemas2 pindah kamar dgn alas salah 
kamar nah tho usir cuy terima minta maaf review sangat penting kamo thanks 
2 jam check out terlalu ketat customer check out telp tdk cek reception cust bukan tanda tangan gl tagih bill msh 


misscom nya kurang kordinasi marketing nya security 
3 pesan single bed besar tp kasi nya 2 twinbed jadi satu tp fasilitas 


4 banyak posting ulas positif kena hotel akhir putus nginap sini dgn keluarga hari tp betapa kaget anak2 kolam 
renang nyata pakai krn sdg maintenance tdk info sblmnya cek in kolam tdk spt umum ajak anak2 inap hotel 
timbang utama kolam renang yg nyaman buat pakai pdhl harga yg beri sesuai dgn promo masuk fasilitas kolam 
renang lapor resepsionis cuma bilang maaf pakai besok siang baru awal cek in sdh kecewa buat bad mood 





5 hotel baru bersih sayang layan prihatin request kamar 1 bed kasih 2 bed jadi 1 ga konfirmasi udah info late 
check out ga info kunci terprotect turun lantai 6 kata receptionistnya nyata naik eh jadi 2 kali dikerjain 
recptionist check out receptionistnya ga minta maaf pura2 tau woo hotel baru berani begini 


Previous 2 3 4 5 ... 69 Next 


Figure 9. Interface of end-user analysis detail page 
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Table 2. Testing the login page (login) Table 3. Data upload test (data validation) 
Field Input Output Annotation Field Input Output Annotation 
Username & Data != Login Denied Rawdata (.csv) IsNull Upload Pending Denied 
Password DB Failed Rawdata (.csv) Not Null | Upload Success Granted 
Username & Data == Login Gis 
Password DB Success 


Table 5. Deletion test of the connecting word 


Table 4. Stemming testing (sentence correction) (stopword) 
Field Input Output Annotation Field Input Output Annotation 
Netizen Array[i] == Netizen Array[i] == 
Comment Words with Stemming Processed Comment (array Words with Processed Deleted 
(array) affix stemmed) affix 
sa jes . Te 
Netizen Array[i] ! Not Not Netizen Array[1] ! Not Not 
Comment Words with Comment (array Words with 
: Stemming Processed Processed deleted 
(array) affix stemmed) affix 


The data upload test carried out on the system is done by performing the data upload functional test. 
The system is functional as expected, the results are shown in Table 3. The next test is carried out by testing 
the stemming and stopword data preprocessing analysis, in which the results are in accordance with the 
results as shown in Table 4 and Table 5. The next step is to conduct a test on the function of database access. 
Table 6 shows the results obtained. The result of functional database access on the system is as it should be. 
Likewise, the test done on the functional database access of the end-user report page shows that the system 
language is running properly, as it is shown in Table 7. 


Table 6. Testing the home page (home) Table 7. Testing report pages (report) 
Field Input Output Annotation Field Input Output Annotation 
Hotel data Array[i] == Data Displays hotel Data analysis Arrayfil Dina Show hotel 
from database Exist Shown data from the —- Exist ea Pas 
Hotel data Array[i] != Data not Do not display database B y 
from database Exist shown hotel data Data analysis ; Do not 
from the Array l) display hotel Donor SO’ 
l= Exist hotel analysis 


database data 


3.5. Limitation and future research 

SentiHotel has shortcoming in the applied model (neural network), especially in the not yet optimal 
and significantly high level of accuracy. This increase in accuracy can be answered by conducting advanced 
experiments to get the terabit model with the best classification accuracy rate. SentiHotel has not been given 
a feasibility study assessment by end users, namely tourists who need assessment information and require 
hotel service information. The assessment has not yet been carried out and requires further experiments, 
especially in terms of analyzing the use of system acceptance by users. The usability system method that can 
be carried out for further research can be done using, the TAM, UTAUT, and other methods [27] so that 
SentiHotel is ready to be used and implemented to provide information to users regarding the assessment of a 
hotel service. 


4. CONCLUSION 

An application called SentiHotel is proposed and developed to apply a model obtained with the best 
performance. The applied neural network optimization model has enabled the system to functionally have 
different capabilities, namely being able to carry out the sentiment process of hotel service review 
assessments which has an accuracy rate of 88.99%. This study provides new contributions based on other 
previous studies where the development of a new application of sentiment analysis based on the neural 
network model is proposed thus it provides benefits for anyone who wants to get a hotel service 
recommendation. The sentiment application analysis of hotel services functionally makes it easy for users, 
especially tourists who are looking for the best hotel reference. For further research, this application requires 
a continuous experimental effort that will lead to the may be even better results. The difficulty of 
implementing a model which contain an algorithm, provides its own level of difficulty; therefore 
collaboration efforts with programmers are very important. 
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